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Dissociating motoric and motivational effects of pharmacological manipulations on operant behavior is 
a substantial challenge. To address this problem, we applied a response-bout analysis to data from rats 
trained to lever press for sucrose on variable-interval (VI) schedules of reinforcement. Motoric, 
motivational, and schedule factors (effort requirement, deprivation level, and schedule requirements, 
respectively) were manipulated. Bout analysis found that interresponse times (IRTs) were described by a 
mixture of two exponential distributions, one characterizing IRTs within response bouts, another 
characterizing intervals between bouts. Increasing effort requirement lengthened the shortest IRT (the 
refractory period between responses). Adding a ratio requirement increased the length and density of 
response bouts. Both manipulations also decreased the bout-initiation rate. In contrast, food 
deprivation only increased the bout-initiation rate. Changes in the distribution of IRTs over time 
showed that responses during extinction were also emitted in bouts, and that the decrease in response 
rate was primarily due to progressively longer intervals between bouts. Taken together, these results 
suggest that changes in the refractory period indicate motoric effects, whereas selective alterations in 
bout initiation rate indicate incentive-motivational effects. These findings support the use of response- 
bout analyses to identify the influence of pharmacological manipulations on processes underlying 
operant performance. 
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Since Skinner (1938/1991), the rate at 
which an operant is emitted has served as the 
principal measure of the effectiveness of 
reinforcement (Killeen & Hall, 2001, but see 
Hursh & Silberberg, 2008). Free operant 
response rate, however, is not a unitary phe¬ 
nomenon. Under most contingencies of rein¬ 
forcement, operant performance seems to be 
organized in bouts separated by relatively long 
pauses. Bout-like organization is evident in 
break-and-run patterns observed under fixed- 
ratio (FR) schedules of reinforcement (Felton 
& Lyon, 1966), fixed-interval (FI) schedules 
(Schneider, 1969), and peak timing procedures 
(Church, Meek & Gibbon, 1994; Sanabria, 
Thrailkill & Killeen, 2009). Bout-and-pause 
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patterns are more readily visible in variable- 
ratio (VR; Reed, 2011) and variable-interval 
(VI) schedules (Conover, Fulton, & Shizgal, 
2001; Shull, Gaynor & Grimes, 2001). These 
patterns suggest that response rate is constitut¬ 
ed by two underlying rates—the rate at which 
bouts are initiated, and the rate at which 
responses are emitted while in a bout. 

Shull and colleagues (Shull et al., 2001, 
2002; Shull, 2004) have shown that the 
components of operant response rate in VI 
schedules, bout-initiation rate and within-bout 
response rate, are differentially sensitive to 
various experimental manipulations. Bout 
initiation rate, but not within-bout response 
rate, positively covaries with rate of reinforce¬ 
ment and level of deprivation (Shull et al, 
2001, 2002; Shull, 2004). The addition of an 
unsignaled ratio requirement at the end of a 
scheduled interval increases the probability of 
remaining in a bout after a response (Shull et 
al., 2001; Shull & Grimes, 2003; Shull, Grimes 
& Bennett, 2004). The rate of initiation and 
the length of bouts decrease with time in 
extinction (Shull et al., 2002). Taken together, 
these results suggest that, under VI schedules, 
(1) bout initiation rate depends primarily on 
rate of reinforcement and deprivation, and (2) 
within-bout response rate is sensitive to sched- 
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ule demands such as tandem ratio require¬ 
ments. 

The differential sensitivity of the compo¬ 
nents of response rate to motivational and 
schedule manipulations (e.g., deprivation level 
and tandem ratio requirement, respectively) 
suggests that important psychological process¬ 
es may be dissociated in nonhuman animals 
on the basis of critical aspects of operant 
performance. Such capability has significant 
implications in behavioral neuroscience, be¬ 
cause the isolation of psychological processes 
is a prerequisite to the identification of their 
underlying neural processes and structures. In 
this regard, it would be ideal if the bout-and- 
pause analysis could be extended to dissociate 
motoric effects in addition to its dissociation of 
motivational and schedule effects. Indeed, 
motoric effects are of great concern in 
biopsychological research, because they are 
often confounded with changes in motivation 
(Avila et al., 2009; Salamone, Correa, Farrar, 
Nunes, Pardo, 2009; we refer to the experi¬ 
mental variation in these effort requirements 
as effort or motoric manipulations). 

The present study aimed to replicate the 
schedule and deprivation-level effects demon¬ 
strated by Shull and colleagues (2001, 2002, 
2004), and to expand upon them by investi¬ 
gating the effect of effort requirement on 
response rate. We manipulated effort by 
varying the height and force requirement of 
the levers, and motivation for food by imple¬ 
menting a 24-hr deprivation period. Our goal 
was to use the bout-and-pause analysis to 
characterize the differential effects of motiva¬ 
tional (deprivation level), motoric (lever 
height and force requirement), and schedule 
(presence/absence of tandem FR require¬ 
ment) manipulations on response rate, and 
to identify performance parameters (bout- 
initiation rate, within-bout response rate, 
etc.) that are differentially sensitive to these 
manipulations. Identifying such effects is 
critical to establish a means of determining 
whether changes in response rate caused by a 
drug are due to an alteration in the motivation 
for the reinforcer or due to changes in motor 
capacity. To avoid interactions between the 
treatment drug and the reinforcer—in partic¬ 
ular if the reinforcer is another drug—tests are 
often conducted under extinction conditions 
(Fuchs, Tran-Nguyen, Specio, Groff & Neise- 
wander, 1998; Stewart & de Wit, 1987). We 


therefore introduced a novel, dynamic model 
to account for changes in model parameters 
during extinction. 

A Bout-and-Pause Model of VI Performance 

In a recent series of studies, Shull and 
colleagues (Shull & Grimes, 2003; Shull, 2004; 
Shull et al., 2004) analyzed the distribution of 
VI interresponse times (IRTs) using log 
survival analyses, and found evidence that such 
a distribution is more accurately described as a 
mixture of two exponential distributions—one 
characterized by very short, burst-like IRTs, 
and the other characterized by much longer 
IRTs. This is consistent with the view that a rat 
can be in either one of two states during a 
session—an engaged or a disengaged state 
(see also Heyman, 1988). When the rat is in 
the engaged state, it responds at a relatively 
high rate ( w , within-bout response rate) 
according to a Poisson process, generating 
the exponential distribution of short IRTs. 
After each response there is a constant 
probability (1 — q) that the rat will remain in 
the engaged state and continue to respond at a 
high rate. There is a complementary probabil¬ 
ity ( q ) that the rat will quit the engaged state 
and enter the disengaged state. In the disen¬ 
gaged state, the rat may perform responses 
incompatible with the target response (e.g., 
locomotion, grooming, etc.) or may simply be 
quiescent for a period of time. Once that 
period of time elapses, the rat will revisit the 
target operandum to emit a response, thus 
reentering the engaged state. The target 
responses that terminate the disengaged state 
are therefore called bout-initiation responses. 
The exponential shape of the distribution of 
intervals between bouts implies that bout- 
initiation responses are governed by another 
Poisson process, albeit one with a much lower 
rate ( b , bout-initiation rate) than the process 
that controls within-bout responses. Thus, 
there is evidence that operant responding is 
not a unitary process, but instead occurs in 
distinguishable bouts, and that overall re¬ 
sponse rate is actually a composite of two 
classes of responses—fast within-bout respons¬ 
es and slower bout-initiation responses. 

According to this model, the probability 
density of an IRT of length t, p(IRT = t), in VI 
schedules of reinforcement can be described 
by the following bi-exponential distribution 
(Shull & Grimes, 2003; Shull, 2004; Shull 
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et al., 2004): 

p(IRT=x) = (\-q)we~ m + qbe- b \ (1) 

where w is the mean within-boul response rate; 
b is the mean bout-initiation rate, and 1 — q is 
the proportion of IRTs that are emitted during 
a bout; q is therefore the proportion of IRTs 
that separate bouts (which is the same as the 
probability of quitting the engaged state after a 
response). Overall response rate is thus a 
composite process controlled by three param¬ 
eters, q, w, and b. We call this model the bi¬ 
exponential model. Note that setting q to zero (or 
one) reduces Equation 1 to an exponential 
distribution, which would be a good descrip¬ 
tion of operant performance not organized in 
groups of bouts. 

The Refractory Period 

Equation 1 assumes that after a response is 
made, an animal can instantaneously make 
another response (i.e., there is no dead time 
between responses). This is likely to be incor¬ 
rect— physical limitations impose ceilings on 
response rates (Killeen, 1994; Killeen & Sito- 
mer, 2003), which yield a dead, or refractory, 
period after each response during which the 
animal cannot make another response (Killeen, 
Hall, Reilly, & Kettle, 2002). This refractory 
period, which we call §, is longer for responses 
that take longer to complete, probably like 
those that involve a higher workload. We 
include § in Equation 1 as 

p(IRT = t|t< 8) = 0 

p(IRT = x\x>8) = (1 - q)we~ w(% ~ ? ' ) + qbe~ b{T ~ s) . 

( 2 ) 

We call Equation 2 the refractory bi-exponential 
model. Note that fixing 5 at zero in Equation 2 
reduces it to Equation 1. Figure 1 provides a 
schematic diagram of the refractory bi-expo- 
nential model underlying Equation 2. 

Predictions 

Prior data suggest that food deprivation 
covaries with the rate at which response bouts 
are initiated, but not with response rates 
within bouts in VI schedules (Shull et al., 
2001, 2002; Shull, 2004). Accordingly, we 
anticipated that depriving rats of food would 
increase b (bout-initiation rate), but not w 
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Fig. 1. A diagram of the refractory bi-exponential 
model of operant performance. The lever press requires 
time 8 to complete. Following a response, the rat either 
remains in the engaged state with probability 1 —q and 
responds on the lever at rate w, or exits the engaged state 
with probability q and returns at rate b. 

(within-bout response rate) or q (probability of 
quitting an engaged state after a response), in 
the distribution of food-reinforced VI IRTs. In 
contrast, prior data suggest that appending a 
tandem FR requirement at the end of the VI 
schedule, which increases the response re¬ 
quirement without substantially affecting rate 
of reinforcement, mainly increases within-bout 
response rates and not bout-initiation rates 
(Shull et al., 2001, 2004; Shull & Grimes, 
2003). Accordingly, we anticipated that the 
tandem FR requirement would increase w, 
possibly decrease q, and would not influence b. 

Skjoldager, Pierre and Mittleman’s (1993) 
study provides the most relevant precedent to 
the motoric manipulation implemented in the 
present study. They found that increasing the 
lever height and force requirement prolongs 
pre-ratio pauses (i.e., reduces rates of engage¬ 
ment) and reduces run rates in a progressive 
ratio schedule. It is unclear, however, the extent 
to which the latter effect might have resulted 
from the longer time it takes to complete more 
effortful responses, which is reflected in the 
refractory period § following each response. We 
thus anticipated that raising the lever and 
increasing its force requirement would de¬ 
crease b, increase §, and possibly decrease w. 

METHOD 

Subjects 

Six Sprague Dawley rats ( Rattus norvegicus, 
designated 505, 507, 517, 519, 520, and 521) 
naive to operant conditioning experimenta¬ 
tion served as subjects. They were approxi¬ 
mately 90 days old and weighed about 350 g at 
the start of the study. They were housed 
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individually on a 12:12 hr reverse light/dark 
cycle with lights on at 1900 hr, and had free 
access to food and water in their home cage, 
unless noted otherwise. 

Apparatus 

Two experimental chambers (305 mm long, 
241 mm wide, and 210 mm high) housed in 
light- and sound-attenuating compartments were 
used. The front and rear walls and the ceiling of 
each experimental chamber were made of clear 
plastic; the front wall was hinged and functioned 
as a door to the chamber. The floor consisted of 
thin metal bars positioned above a catch pan. In 
the horizontal middle of a side wall (perpendic¬ 
ular to the front and rear walls) was a square 
aperture (51 mm sides, 15 mm from the 
chamber floor) to the receptacle for 45-mg 
sucrose pellets (dustless precision formula 
F0042; Bio-Serv, Frenchtown, NJ). The chambers 
were equipped with two retractable levers (MED 
associates, ENV-112CM) mounted flanking the 
access to the pellet receptacle. The inside edge of 
each lever was 8 mm from the closest vertical 
edge of the receptacle. 

One lever (the “low” lever) was located 
21 mm above the floor; the other lever (the 
“high” lever) was 165 mm above the floor and 
16 mm below the ceiling. Rats could press the 
low lever but not the high lever without 
rearing. Force activation requirements for the 
low and high lever were 0.05 N and 0.78 N, 
respectively, except during autoshaping. Force 
requirements were measured from the tip of 
the lever using a stylus force gauge and 
manipulated by adjusting the spring tension 
of each lever. Flenceforth the high height/ 
force lever will be referred to as the high 
workload lever and the low height/force lever 
will be referred to as the low workload lever. The 
assignment of high versus low workload to the 
lever closest to the door was counterbalanced 
across chambers and remained fixed for the 
duration of the experiment. Extraneous noise 
was masked by a ventilation fan in each 
chamber. There was no illumination in the 
chambers during sessions. Data collection and 
experimental events were handled by MED- 
PC 1M software and hardware. 

Procedure 

Magazine training and autoshaping. Each sub¬ 
ject completed one session of magazine train¬ 


ing, in which 60 sucrose pellets were deli¬ 
vered, response-independently, 1 every 60 s. 
At the end of the training session, it was 
verified that all rats ate all of the pellets. Nine 
sessions of autoshaping were then conducted. 
Each autoshaping trial started with the exten¬ 
sion of a lever, which was retracted after 8 s or a 
lever press, whichever happened first; lever 
retraction was followed by the delivery of one 
pellet. The intertrial interval (ITI) was 48 s. 
During autoshaping, both levers were set at the 
“low” height; lever force activation require¬ 
ments were both set at 0.15 N. For all other 
phases of the experiment the lever work 
requirements were as described in the appara¬ 
tus section. 

Phase 1: Maintenance. Daily experimental 
sessions were conducted following the auto¬ 
shaping phase. Each session began with a 5- 
min acclimation period in which no experi¬ 
mental events occurred. The remainder of the 
session was divided into trials, each signaled by 
the extension of only one of the two levers. 
The lever closest to the chamber door was 
always extended for the first trial of each 
session. The extended lever (high or low) was 
strictly alternated between trials. Each trial 
ended and the lever was retracted when a 
sucrose pellet was delivered or when 300 s had 
elapsed since the start of the trial, whichever 
occurred first. Trials were separated by a 15-s 
ITI during which both levers were retracted. 

All experimental sessions lasted for 1 hr or 
60 trials, whichever occurred first. Sessions 
ended only after the end of a trial, never 
during. Sessions were conducted 7 days a 
week. 

Table 1 lists the experimental conditions in 
the order in which they were presented. Each 
condition is identified by the schedule on 
which pellet deliveries were programmed. 
During continuous reinforcement (FR 1), 
pellets were contingent on a single lever press. 
During VI schedules, pellets were contingent 
on the first lever press following an unsignaled 
interval that was randomly sampled without 
replacement from a flat 48-item distribution of 
intervals, which had a mean of the stipulated 
VI duration and ranged between 5 and 240 s. 
The tandem variable-time (VT) 120 s fixed- 
ratio 5 (tandem VT 120-s FR 5) schedule was 
similar to a simple VI 120-s schedule, but 
pellets were contingent on the fifth lever press 
following the interval. 
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Table 1 


Number of sessions in training and experimental 
conditions, arranged in chronological order. 


Schedule of Reinforcement 

Sessions 

Phase 1 

FR 1 

i 

VI 5 s 

i 

VI 10 s 

i 

VI 20 s 

i 

VI 30 s 

n 

VI 120 s 

20 

Tandem VT 120-s FR 5 

27 

Tandem VT 120-s FR 5 with food deprivation 

1 

Phase 2 

Tandem VT 120-s FR 5 

7 

Extinction 

1 


Acute food deprivation was instated imme¬ 
diately after the 27 th tandem VT 120-s FR 5 
session. Chow was removed from the rats’ 
homecage for 24 hr (± 1 hr). One session was 
conducted on the tandem VT 120-s FR 5 s 
schedule under food deprivation. Food depri¬ 
vation was terminated immediately after this 
session to minimize the possibility of chronic 
food-restriction effects on operant perfor¬ 
mance (Epstein, Leddy, Temple & Faith, 
2007). 

Phase 2: Extinction. Performance was restabi¬ 
lized on the tandem VT 120-s FR 5 schedule 
for seven sessions, after which one extinction 
session was conducted. During extinction, only 
the lever closest to the door was extended after 
the initial acclimation period; it remained 
extended for the entire session, with no 
consequential pellet delivery. The extended 
lever (high or low workload) was counterbal¬ 
anced across rats, with 3 rats exposed to each 
lever. 

Data Analysis 

Statistical analyses of overall response rates 
and parameters of the quantitative model were 
conducted using a within-subject 2X3 
(workload X condition) ANOVA. The two 
levels of the workload factor were low and high 
workload. The three levels of the condition 
factor were VI 120 s (VT), tandem VT 120-s FR 
5 without food deprivation (Tandem), and 
tandem VT 120-s FR 5 with food deprivation 
(Food Dep). Dependent measures for each level 
of the condition factor were the average of the 
last 4 VI sessions, the average of the last 4 


Tandem sessions, and the only Food Dep session. 
Because the condition factor was a within- 
subject factor with more than two levels, 
Mauchly’s test of sphericity of the covariance 
matrix (Mauchly, 1940) was applied. This test 
verified the homogeneity of variance of the 
difference scores because violating variance 
homogeneity can inflate Type I error rates 
(Myers & Well, 1995). For terms that violated 
the sphericity assumption, their degrees of 
freedom were reduced using the Huynh-Feldt 
epsilon (Huynh & Feldt, 1970), which coun¬ 
tered Type I error rate inflation. When the 2 X 
3 ANOVA revealed significant main effects or 
interactions, paired-samples /-tests (pairing 
within-subject) were conducted on compari¬ 
sons of interest. More specifically, when an 
effect involving the condition factor was found 
to be significant, follow-up paired-samples 
t-tests were conducted only between VT versus 
Tandem (to assess the effect of the tandem 
ratio requirement) and Tandem versus Food. Dep 
(to assess the effect of food deprivation), and 
not between VI and Food Dep. The times at 
which responses were emitted during the 
single extinction session were also collected, 
but no between-subject statistical inferences 
were made. 

RESULTS PHASE 1: MAINTENANCE 
Reinforcement and Response Rate 

Panel A in Figure 2 shows daily mean 
reinforcement rates on both low and high 
workload levers for each manipulation. Panel 
B in Figure 2 shows the mean reinforcement 
rates averaged over the last four training 
sessions in the VI and Tandem manipulations 
and the mean reinforcement rates from the 
single Food Dep manipulation. Reinforcement 
rate did not appear to be affected by either 
workload or condition: ANOVA failed to 
detect a main effect of workload, /•'( 1, 5) = 
1.74, p > .20, condition, F < 1, NS, or a 
workload X condition interaction, F < 1, NS. 
This indicates that our schedule successfully 
controlled for reinforcement rates on both 
levers across schedule/deprivation manipula¬ 
tions. 

Panel C in Figure 2 shows daily mean 
response rates during VI, Tandem, and Food 
Dep sessions. Panel D in Figure 2 shows mean 
response rates averaged over the last four VI 
and Tandem sessions and the mean response 
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Fig. 2. Mean (± SEM) reinforcement rate (Panels A and B) and overall response rate (Panels C and D) for 
experimental conditions in Phase 1. Left panels show daily means for low and high workload levers. Right panels show 
means averaged over the last four sessions for 17 and Tandem condition and the mean for the single Food Dep session. 
Asterisks indicate significant (p < .05) effects of condition (schedule/deprivation manipulations). 


rates from the single Food Dep session for each 
lever. A NOVA found no significant condition 
X workload interaction effect on response 
rates, F < 1, NS. ANOVA also found no 
systematic differences between response rates 
on the high versus low workload lever (main 
effect of workload, F < 1, NS). In contrast, 
ANOVA found a main effect of condition, F( 2, 
10) = 18.55, p < .01. A follow-up 7-test 
comparing VI versus Tandem found that 
subjects responded more during the Tandem 
condition than during the VI condition, 7(5) = 
3.85, p < .02 (left asterisk in Figure 2D). A 
second follow-up 7-test comparing Tandem 
versus Food Dep found that rats also responded 
more during Food Dep than during Tandem, 
7(5) = 3.87, p < .02 (right asterisk in 
Figure 2D), in agreement with the hypothesis 
that motivation is enhanced by increased food 
deprivation. 

Model Selection 

Although statistical analysis suggested that 
response rate was not affected by workload, 
variability in performance between and within 
subjects may have obscured systematic changes 
in more fine-grained parameters that underlie 


overall response rates. We further explored 
this possibility by considering four variations of 
a model of IRT distributions (Equation 2, 
Figure 1): a single-exponential model, where q 
= 0 and § = 0; a refractory exponential model, 
where q = 0 but 8>0;a bi-exponential model, 
where q > 0 but § = 0 (Equation 1), and a 
refractory bi-exponential model, where q > 0 
but 5 > 0 (Equation 2). 

We fitted each model to each rat’s daily 
IRT data on each lever, using the method of 
maximum likelihood (Myung, 2003). The 
maximum likelihood estimate (MLE) of each 
model was the product of the model’s daily 
likelihood estimates across levers and sub¬ 
jects. Akaike Information Criterion (AIC; 
Burnham & Anderson, 2002; see Appendix 
A for a brief explanation of the maxi¬ 
mum likelihood method, AIC, and AAIC) 
was then used to evaluate the relative 
goodness of fit of each model. In the present 
experiment, a trial occasionally timed out 
before the animal had earned a reinforcer, 
thus yielding periods that ended without a 
response. Appendix B provides the expres¬ 
sion for the probability that the animal 
does not emit a response for a given duration 
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Table 2 

Minimum daily AAIC for each of four models of operant performance, for each experimental 
condition in Phase 1. 


Model 

VI 

Tandem 

Food Dep 

Single exponential ( w) 

3377 

6239 

25497 

Refractory exponential ( w , 6) 

2836 

5771 

22744 

Bi-exponential ( q, w, b) 

1097 

1700 

9622 

Refractory bi-exponential ( q, w, b, 8) 

0 

0 

0 


Note. Computation of AAIC is explained in Appendix A. Free parameters are indicated in parenthesis following model 
names. Note that the food deprivation condition consisted of only a single session. 


for the most general model (i.e., refractory 
bi-exponential). Parameters were estimated 
for each of the four models, for each animal, 
in each experimental condition, based on 
the joint probability of all IRTs and the “no¬ 
response” periods. 

Table 2 shows the minimum AAIC across 
daily sessions under each condition for each 
model. The refractory bi-exponential model 
(Equation 2) was consistently the best model 
on every training session. The finding that the 
next smallest AAIC across all sessions was 1097 
means that the refractory bi-exponential mod¬ 
el was at least e 548 times more likely than the 
next best model (nonrefractory bi-exponential 
model) on any given day. 

To illustrate the goodness of fit of the 
selected model, Figure 3 shows log survival 
plots of IRTs (continuous curves) of a typical 
rat across experimental conditions, along with 
fitted refractory exponential (dotted) and 
refractory bi-exponential (dashed) traces. All 
plots took the approximate shape of a broken 
stick: Many IRTs were very short (steep portion 
of the curve on the left side of each plot); the 
longer IRTs ranged over much longer intervals 
(flatter portion of the curve on the right side 
of each plot). The refractory bi-exponential 
model fitted the data adequately, and certainly 
better than the single exponential model. The 
fitted nonrefractory bi-exponential trace (not 
shown) was almost identical to the refractory 
bi-exponential trace, except that the former 
was shifted to the left by 0.1 s. The similarity of 
both traces highlights the advantage of using a 
likelihood-based analysis: AIC clearly showed 
that the inclusion of the refractory period 
provided a better description of the data. This 
advantage would have been missed by both 
visual inspection of the survival function and 
by fitting a curve to the function using the 


method of least squares (Kessel & Lucke, 
2008). 

Effects on Parameter Estimates 

To assess the effect of workload, schedule of 
reinforcement, and food deprivation on re¬ 
fractory bi-exponential parameters, we com¬ 
pared estimates of these parameters 1 across 
experimental manipulations. This comparison 
was based on the same 2X3 (workload X 
condition) ANOVA used to analyze overall 
response rates. The two levels in the workload 
factors were low and high workload, and the 
three levels in the condition factor were VI, 
Tandem, and Food Deprivation (Food Dep ). 
Because our model assumed that each param¬ 
eter was independent from one another, a 
separate ANOVA was conducted for each 
parameter. Estimates for individual rats are 
shown in Appendix C. 

Panels A and B in Figure 4 show the 
probability of quitting a bout, q, for each of 
the two levers. ANOVA found no workload X 
condition interaction effect on q, F < 1, NS. 


1 There were occasional sessions in which one animal did 
not emit bout-like responding—specifically rat 520 on three 
sessions and rat 519 on one session under the VI condition. 
On these sessions, these rats’ AIC scores from the refractory 
single exponential model were the lowest. This was because 
the estimates for q using the bi-exponential model were so 
close to zero that the parameter b did not account for 
enough variance to justify its inclusion. This means that the 
estimates of b were based on few responses and were thus 
unreliable. On these sessions, we therefore used the 
refractory single exponential model for these animals, with 
q = 0, and b for these animals was not estimated. On 
another two sessions under the Tandem condition, rat 520 
emitted only one response on the high workload lever. We 
therefore omitted to analyze this rat’s IRT data from the 
high workload lever entirely on these two sessions. None of 
the above atypical responding occurred during the last four 
sessions in each condition, on which statistical analyses were 
based. 
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VI Condition 





Time, r(s) 

Fig. 3. Log survival plots of IRTs produced by a representative rat in all experimental conditions of Phase 1 
(continuous curves). Left and right panels show data from low and high workload levers, respectively. The maximum 
likelihood fits of the refractory single exponential (Ex) and bi-exponential (Bi-ex) models are also shown. The best fitting 
parameters for the two models are displayed in each graph, q. proportion of IRTs separating bouts; w (responses/sec): 
within-bout response rate; b (responses/sec): bout-initiation rate; 8 (sec): refractory period. The rat was selected by 
ranking the overall response rate on each lever in the last session of each condition in Phase 1 for each rat, then 
averaging the rank across levers and conditions, and selecting the rat with the third highest average rank. 
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Fig. 4. Mean (± SEM) bi-exponential model parameters: probability of quitting a bout, q (Panels A and B); the 
refractory period, 5 (Panels C and D); within-bout response rate, w (Panels E and F); and bout initiation rate, b (Panels G 
and H). Left panels show daily means for low and high workload levers. Right panels show means averaged over the last 
four sessions for V7and Tandem condition and the mean for the single Food Dep session. Asterisks indicate significant (p < 
.05) effects of condition (schedule/deprivation). Pound signs indicate significant effects of workload. 
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Changing motor workload had no significant 
effect on q [main effect of workload: F( 1,5) = 
1.07, p > .30]. In contrast, ANOVA found that 
q was significantly affected by condition [main 
effect of condition: T(1.0, 5.1) = 19.64, p <. 
01]. A follow-up /-test comparing VI versus 
Tandem found that increasing ratio require¬ 
ment reduced q on both levers, /(5) = 4.54, p 
< .01, (asterisk in Figure 4B). A second follow¬ 
up /-test comparing Tandem versus Food Dep 
found that food deprivation did not affect q 
significantly, Z(5) = 1.63, p > .10. 

Panels C and D in Figure 4 show mean 
estimates of the refractory period, 5, for each 
of the two levers. On average, § was systemat¬ 
ically higher for the high versus the low 
workload lever, although the difference was 
small (pooled average difference = 0.033 ± 
0.009 s) 1 2 * * * * * 8 . ANOVA found no workload X 
condition interaction effect on 5, .F(l.l, 5.3) 
= 1.96, p > .20, but it found a significant 
main effect of workload, F(l, 5) = 15.38, 
p < .02 (pound sign in Figure 4D), showing 
that increasing motor workload increased §, 
as predicted. The main effect of condition 
on § was not significant, T(l.l, 5.3) = 3.12, 
p > .10. 

Panels E and F in Figure 4 show mean 
estimates of the within-bout response rate, w, 

1 Our estimate of 8 carries a slight bias. This is because 

maximum likelihood is achieved when 8 is as large as 

possible, i.e., at the minimum IRT. However, if X is an 

exponentially distributed random variable with rate b and 

no refractory period (8 = 0), and if we take n independent 
samples from X (e.g., n IRTs from a subject), then 

miii{,Y/,.... X n ) will also be exponentially distributed with 
mean 1 /nb (Ross, 2007). Similarly, given a bi-exponential 
distribution with no refractory period, if we take ( l—q)n 
samples from the exponential distribution with rate w and 

qn samples from the other exponential distribution with 
rate b, then we expect the minimum to be exponentially 
distributed with mean 1 /[(1—q) nw+qnb], This is therefore 
our bias when we use the minimum IRT as our estimate for 

8 in the refractory bi-exponential model. We took bi¬ 
exponential parameter estimates from the last four 
sessions under the VI and Tandem conditions and from 
the single Food Dep session, and substituted them into the 
equation l/[(i— q) nw+qnb], where n is the number of 
responses emitted by the animal. The average bias pooled 
across conditions and levers was 0.005 ± 0.003 s, and there 
was no substantial difference between the biases on the 
two levers (0.002 ± 0.002 s). The bias was only —4% of the 
average estimate for 8 (i.e., minimum IRT; 0.11 ± 0.01 s) 
and was also much smaller than the average difference in 8 
between the high and low workload levers (0.033 ± 
0.009 s). Therefore the bias in the present study was too 
small to affect any of the findings, and was consequently 
ignored. 


for the two levers. ANOVA found no signifi¬ 
cant workload X condition interaction effect 
on w, F < 1, NS. ANOVA found that the main 
effect of workload was not significant at the .05 
level, F{\, 5) = 4.50, p > .08, even though 
differences in mean w between workloads may 
hint that high workloads maintained higher w. 
ANOVA found a significant main effect of 
condition on w, F( 2, 10) = 9.45, p < .01. A 
follow-up /-test comparing VI versus Tandem 
found that w increased when ratio require¬ 
ment was increased, /(5) = 4.49, p < .01 
(asterisk in Figure 4F). A second follow-up 
/-test comparing Tandem versus Food Dep found 
that food deprivation did not affect w, /(5) = 

0.92, p > .35. 

Panels G and H in Figure 4 show mean 
estimates of the bout initiation rate, b, for the 
two levers. ANOVA found no significant 
workload X condition interaction effect on b, 
F < 1, NS. ANOVA found a significant main 
effect of workload on b, F( 1,5) = 19.83, p < 
.01, with b being lower on the high workload 
lever (pound sign in Figure 4H). ANOVA also 
found a marginally significant main effect of 
condition at the .055 level, F( 1.2, 6.0) = 5.48, p 
= .054. A follow-up /-test comparing VI versus 
Tandem found that b was significantly lower 
under the Tandem condition, /(5) = 3.40, p < 
.02 (left asterisk in Figure 4H), suggesting that 
animals initiated response-bouts less frequent¬ 
ly on both levers after ratio requirement 
increased. A second follow-up /-test comparing 
Tandem versus Food Dep found that b was 
increased by food deprivation, Z(5) = 3.19, p 
< .03 (right asterisk in Figure 4H), in agree¬ 
ment with the hypothesis that food deprivation 
enhances motivation. 

DISCUSSION PHASE 1: MAINTENANCE 

A superficial examination of response rate 
alone indicated that responding on a VI 
schedule increases if the tandem ratio require¬ 
ment increases, or if the animals are deprived 
of food (Figure 2, Panel C and D). Both 
effects, under conditions of constant rate of 
reinforcement (Figure 2, Panel A and B), are 
replications of well-demonstrated phenomena 
(Ferster & Skinner, 1957; Skinner 1938/1991; 
Shull et al., 2001). Interestingly, no effect of 
workload on overall response rate was detect¬ 
ed. This result is inconsistent with many 
reports indicating an inverse relationship 
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between force requirements and response rate 
(Adair & Wright, 1976; Ailing & Poling, 1995; 
Bradshaw, Szabadi & Ruddle, 1983; Chung, 
1965; Posadas-Sanchez, 2005), although sup¬ 
port for this relationship is not unequivocal 
(Collier & Jennings, 1969; Elsmore & Brown- 
stein, 1968; Stanley & Aamodt, 1954; Zarcone, 
Chen & Fowler, 2007, 2009). Nonetheless, 
overall response rate is a crude measure of 
performance: By collapsing all the IRTs into 
the denominator, response rates neglect the 
information contained in the distribution of 
IRTs. We suspected that changes in IRT 
distribution across conditions would uncover 
interesting effects, including those produced 
by changes in workload. 

Our first task was to determine the most 
appropriate characterization of IRT distribu¬ 
tions. We thus considered four models, three 
of which were simplified versions nested within 
a fourth, more complex model—the refractory 
bi-exponential model (Equation 2). The com¬ 
plexity of this model was well justified by the 
variance it accounted for and, therefore, it was 
adopted. The refractory bi-exponential model 
assumes that responses occur in bouts; it 
consists of four parameters: q (the probability 
of quitting a response bout; its complement, 
1 — q, is the probability of continuing in a 
bout), § (the minimum IRT), w (the rate of 
responding within a bout), and b (the rate of 
bout initiation). 

Finally, we identified substantial changes in 
refractory bi-exponential parameters caused by 
changes in experimental conditions. Consis¬ 
tent with Shull et al. (2001), we found that the 
tandem VT FR schedule maintained higher 
response rates than the simpler VI schedule 
because it sustained longer bouts of fast 
responding (lower q, higher w ), even though 
these bouts were less frequent (lower b ). Food 
deprivation selectively increased bout frequen¬ 
cy, thus yielding even higher response rates; 
this also replicated Shull’s (2004) findings. We 
also found that mean response rates obscured 
three mutually compensating effects of work¬ 
load: Higher workloads yielded less frequent 
bouts (lower b) of responses that took longer 
to complete (higher 8), but that tended to be 
emitted at higher rates (higher w). Although 
not statistically significant, the workload-in¬ 
duced change in wwas sufficient to counteract 
the depressing influence of b and 8 on overall 
response rate. 


Among the factors considered in this study, 
changes in schedule of reinforcement affected 
only the probability of staying on the lever (1 — 
q) and the rate of within-bout responding ( w ), 
whereas workload uniquely affected the min¬ 
imum IRT (8). Thus, evidence presented here 
supports the use of q and w as indices of 
schedule effects, and 8 as an index of motoric 
effects in food-maintained behavior. Motiva¬ 
tional effects, such as those of food depriva¬ 
tion, may be identified by exclusive changes in 
the rate of bout initiation b. This means that 
changes in b that are accompanied by changes 
in other parameters may be caused by non- 
motivational manipulations. As shown in Fig¬ 
ure 4 Panel H, a schedule manipulation 
(tandem FR requirement) and a motoric 
challenge (higher workload lever) yielded 
reductions in b. These nonmotivational ma¬ 
nipulations also influenced other parameters; 
only food deprivation influenced b alone. 

RESULTS PHASE 2: EXTINCTION 

Figure 5 shows, in separate plots for low and 
high workload, the cumulative lever presses 
emitted by individual rats and the time when 
half of those lever presses were emitted during 
the extinction session. On average, fewer 
responses appear to be emitted during the 
extinction of high workload lever pressing, 
although the variability between subjects pre¬ 
cludes any meaningful statistical analysis. In 
contrast, the rate at which responding decayed 
during extinction, indexed by the half-life of 
the cumulative response, was similar across 
workload levels. As in Phase 1, we anticipated 
that the analysis of response aggregates pro¬ 
vided no more than a general semblance of 
the extinction process. We thus applied the 
modeling exercise from Phase 1 to the data 
from Phase 2. 

Because the exponential model is a special 
case of the bi-exponential model, we focused 
on generalizing the latter model to extinction 
performance. Like the maintenance model, 
the extinction model assumes that responses 
occur stochastically and independently accord¬ 
ing to two underlying independent Poisson 
processes—one with high rate (within-bout) 
and one with low rate (bout initiation). It is 
assumed that, during an extinction session, 
one or more of the following parameters decay 
exponentially towards zero over time: the 
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Fig. 5. Cumulative lever presses emitted during extinction on the low and high workload levers for individual rats. 
Vertical drop lines indicate the time at which each subject emitted half of its total lever presses in the extinction session. 
Rat numbers are indicated at the end point of each record. 


probability of remaining in a bout (1 — q ), the 
within-bout response rate ( w ), and the bout- 
initiation rate ( b ). The decay of any one of 
these parameters would yield longer IRTs, and 
thus reduced response rates, as a function of 
time in extinction. The nomination of these 
decay processes is not motivated by theoretical 
considerations, but provides reasonable de¬ 
scriptions of the extinction process, given the 
refractory bi-exponential model. 

Specifically, let (1 — q 0 ) be the baseline 
probability of remaining in the engaged state 
at the beginning of the extinction session, w 0 
be the baseline within-bout response rate, and 
b 0 be the baseline bout-initiation rate. Let 1 — 
q t be the probability of quitting the engaged 
state at time t into the extinction session, w, be 
the within-bout response rate at time t into the 
session, and b t be the bout initiation rate at 
time t into the session. Then: 

l-q, = (\-qo)e~ yt 

Wt = iu 0 e~ c “ P>a (3) 

b, = k ) e~ ISt 

where y, a, and P are the rates of decay of (1 — 
qo ), Wo, and b 0 respectively. The probability of 
quitting a bout at time t, q t , is simply 1 — (1 — 
q t ). Note that the exponential function is used 
in Equation 3 to describe the decay of 
parameters as a function of time in extinction, 
and not to describe the probability distribu¬ 
tion of IRTs (cf. Equations 1-2). For simplicity, 
we assume that if a rat responded at time t and 
the next response occurs at t + d, then the 
parameters q t , w, and b t are given by Equation 3 
and they remain constant between t and t+d. 


Appendix D provides expressions for the 
probability that the animal does not emit a 
response between its last emitted response and 
the end of the session. Parameters were 
estimated for each animal in each experimen¬ 
tal condition, based on the joint probability of 
all IRTs and the “no-response” periods. 

Because parameters q, w, and b change as a 
function of time in extinction, we call this 
model the dynamic refractory bi-exponential model, 
in contradistinction to the static model of 
Equation 2. Note that if all three decay-rate 
parameters (y, a, P) are set to zero, the model 
reduces to Equation 2. Table 3 lists all the 
parameters of both static and dynamic refrac¬ 
tory bi-exponential models with their mean¬ 
ing, for reference. 

We had no a priori hypothesis as to which of 
the parameters, (l — q 0 ), Wo, or b 0 , would decay 
during the extinction session. We therefore 
used maximum-likelihood estimates and AIC 
to determine the most efficient model. The 
following models were compared using AIC: 
(a) single versus bi-exponential (qo = 0 vs. qo & 
0); (b) nonrefractory versus refractory (8 = 0 
vs. 8 > 0); and (c) all possible combinations of 
decay rate parameters (y, a, and P) fixed at 
zero versus not fixed at zero. Models with 
parameters fixed at zero are more parsimoni¬ 
ous and are analogous to the null hypothesis 
that those parameters are superfluous (Burn¬ 
ham & Anderson, 2002). The AIC scores for 
these “null” models would be lower, and 
hence these models would be favored, if 
alternative models that allow these parameters 
to be free did not account for substantially 
more variance. There are a total of four single 
exponential candidate models: two models 
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Table 3 

List of parameters for the static and dynamic models. 
Static models 

8 Refractory period 

q Probability of quitting a bout 

w Within-bout response rate 

b Bout-initiation rate 

Dynamic models 

8 Refractory period 

1—qo Baseline probability of remaining in bout 

w 0 Baseline within-bout response rate 

b 0 Baseline bout-initiation rate 

y Rate of decay of 1—qo 

a Rate of decay of w 0 

(3 Rate of decay of bo 


with a either free or fixed at zero X two 
models with § either free or fixed at zero. 
There are a total of 16 bi-exponential candi¬ 
date models: 2 3 models with y, a, and p, each 
either being free or fixed at zero, and for each 
bi-exponential model 5 can be free or fixed at 
zero, yielding 2’ X 2 = 16 candidate models. 
Animals were grouped according to which 
lever was presented during the extinction 
session (high vs. low workload); model selec¬ 
tion and parameter estimation were conduct¬ 
ed for each group separately. 

AIC analysis showed that the refractory bi¬ 
exponential model fitted extinction data bet¬ 
ter than both single exponential models and 
the nonrefractory bi-exponential model. Ta¬ 
ble 4 shows the five best fitting models for 
extinction of high versus low workload lever 
pressing, as well as the best fitting nonrefrac¬ 
tory bi-exponential model and the best fitting 
single exponential model. Model names indi¬ 
cate whether the model is refractory (6 is free) 


and the decay rate parameters that are allowed 
to vary freely. Each model’s AAIC is shown in 
parentheses. The best fitting refractory bi¬ 
exponential models were more than e 700 times 
more likely than the best nonrefractory bi¬ 
exponential model, which was in turn more 
than e B50 ° times more likely than the best 
fitting single exponential model. This provides 
strong evidence that bout-like responding 
occurs during extinction, justifying the present 
fine-grained analysis. The likelihood of each of 
the top four models for both groups (extin¬ 
guished on low vs. high workload levers) is 
fairly similar to each other. In fact, P was the 
only decay rate parameter that the present 
data unequivocally suggested needed to be 
free: For both groups of animals, the best 
models with P set to zero (ranked fifth in 
Table 4) were more than e times less likely 
than a similar models with p allowed to be free, 
providing strong evidence that bout-initiation 
rate declined as extinction progressed. Table 4 
also shows that models with y or a set to zero 
(ranked second) had low AAICs. The present 
data therefore did not provide substantial 
evidence that the probability of remaining in 
a bout or the within-bout response rate 
declined during an extinction session (for 
model selection criteria, see Appendix A). The 
model ranked second, underlined in Table 4, 
was therefore selected as best balancing 
parsimony and goodness-of-fit. 

Figure 6 shows how IRTs (pauses between 
responses) changed as a function of time in 
extinction. The x-coordinate of each point is 
the time t when a response was emitted; the y- 
coordinate shows how long the animal waited 
until emitting the next response. Note that the 
y-axis is plotted on a log scale. Also shown are 


Table 4 


Best extinction models according to AIC. 


Model rank 

Low workload (AAIC) 

High workload (AAIC) 

i 

8, a, |3 (0) 

8, a, p (0) 

2 

6 , 3 (0) 

5, P (1) 

3 

6, y, P (2) 

8, y, a, p (5) 

4 

8, y, a, p (2) 

5, y, P (5) 

5 

8, y (250) 

8 (121) 

Best nonrefractory bi-exp 

P (2829) 

P (1483) 

Best single exp 

8, ot (12771) 

8, a (8736) 


Note. Free parameters are listed for each model; 1 — q 0 , w 0 , and b 0 were free to vary for all bi-exponential models (see 
Equation 3). The best nonrefractory and single exponential models are listed for comparison. Selected models are 
underlined (see Appendix A for explanation). 
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Fig. 6. Interresponse times (IRTs) as a function of time t in extinction. The broken, solid, and dotted lines are, 
respectively, traces of average bout initiation IRT (1 /b t ), average within-bout IRT (1 /w t ), and constant minimum IRT (5), 
drawn from the dynamic refractory bi-exponential model selected in Table 4 and fitted using the maximum likelihood 
method. Traces of 1 /b t have been joined by a smooth straight line for illustrative purposes (see main text). 
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Table 5 


Group mean parameter estimates (±SEMs) of extinction 
performance from the selected model, which fixed y and a 
at zero (Model 2 in Table 3). 


Parameter 

Low workload 

High workload 

1o 

0.11 ± 0.02 

0.12 ± 0.00 

5 (s) 

0.073 ± 0.003 

0.083 ± 0.009 

wo (responses/s) 

5.99 ± 1.36 

4.71 ± 0.91 

b 0 (responses/s) 

0.19 ± 0.02 

0.09 ± 0.04 

P (X1(T 3 /s) 

1.21 ± 0.46 

0.71 ± 0.20 


the fitted mean IRTs drawn from the dynamic 
refractory exponential model selected in Ta¬ 
ble 4—solid lines represent within-boul IRTs 
(8 + 1/ w t ) and broken lines represent be- 
tween-bout IRTs (§ + 1/ b t ). Note that although 
the selected dynamic model assumes that b t 
remains constant (flat) between consecutive 
responses and only increases in discrete 
“jumps”, the mean IRTs in Figure 6 have 
been joined by a smooth straight line for 
illustrative purposes. Figure 6 clearly shows 
clusters of rapid within-bout responses with 
mean IRT of about 0.3 s, intermixed with 
much slower bout-initiation responses with 
mean IRT of about 10 s at the beginning 
of the extinction session, and rising exponen¬ 
tially to about 100 s by the end of the session. 
Rat 519 ceased responding completely at 
about 1500 s into the session; all other rats 
showed a more gradual decline in bout- 
initiation rate. 

The group mean parameter estimates 
(±SEMs) of model 2 (Table 4) for each group 
are presented in Table 5. Due to the small 
group size (n = 3), statistical analysis was not 
conducted on parameter estimates. Nonethe¬ 
less, there are some noteworthy qualitative 
inconsistencies and regularities in parameter 
estimates. The difference in w 0 across work¬ 
loads was not in the same direction as observed 
during Phase 1. As the statistical analysis 
suggested in Phase 1, workload-induced 
changes in within-bout response rate do not 
appear reliable. The differences in b 0 and 5 
across workloads, nonetheless, are consistent 
with the differences observed during Phase 1, 
thus confirming that higher workload yields 
fewer bouts of prolonged responses. Absent 
reinforcement, bouts of more laborious re¬ 
sponses do not appear to decline faster in 
frequency. 


DISCUSSION PHASE 2: EXTINCTION 

The present experiment complements a 
previous study by Shull et al. (2002). In their 
study, extinction sessions were separated into 
blocks of approximately 20 min, with the 
assumption that model parameters are con¬ 
stant within a block. Then, two separate 
analyses were conducted to estimate model 
parameters for each block, and changes in 
parameters as a function of block (time in 
extinction) were visually compared. The first 
analysis used log survival plots to fit parameters 
for each block. However, the authors noted 
that this method was problematic because the 
number of IRTs per block became increasingly 
small, which reduced the reliability of param¬ 
eter estimates in the later blocks. The second 
technique imposed an arbitrary cutoff-IRT. All 
IRTs shorter than the cutoff were classified as 
within-bout, and all IRTs longer than the 
cutoff were classified as between-bout. Al¬ 
though Shull and colleagues used this method 
without any reported issues, the selection of 
the cutoff-IRT has the potential to misclassify 
pauses between bouts as within-bout IRTs, and 
vice versa. The present study circumvented 
these difficulties by generalizing the response- 
bout model so that any dynamic changes in 
steady-state parameters can be captured quan¬ 
titatively. This generalization not only allows 
parameters to be estimated for individual 
subjects on a sound theoretical basis, it also 
has the potential to allow different hypotheses 
to be explicitly tested using AIC, such as which 
of the steady-state parameters ( q , w, or b) 
change during extinction. The results from 
the present extinction experiment supported a 
relatively simple extension of the static refrac¬ 
tory bi-exponential model: when reinforce¬ 
ment is discontinued, only the rate of bout 
initiation declines exponentially over time. 
Research elsewhere has shown similar effects 
of extinction on bout initiation, although 
small decrements in bout length were also 
reported (Shull et al., 2002; Podlesnik, Jime- 
nez-Gomez, Ward, & Shahan, 2006). 

Overall, these findings are consistent with 
reports of selective effects of rate of reinforce¬ 
ment on rate of bout initiation (Johnson, 
Pesek, & Newland, 2009; Shull et al., 2001). 
The evidence supports the notion that rein¬ 
forcement operates primarily on the initiation 
of response bouts (Heyman, 1988), although 
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other studies have shown variations in bout 
length (Shull et al., 2004) and within-bout 
response rate (Conover et al., 2001) with rate 
of reinforcement. The constancy of within- 
bout “tempo”, for instance, is particularly 
salient in the flat solid lines of Figure 6. As 
discussed in Phase 1, however, bout initiation 
appears to be sensitive also to workload and 
schedule demands. It is yet unclear whether 
the workload manipulation implemented in 
this study influenced the rate at which bout 
initiation declines during extinction. 

GENERAL DISCUSSION 

Our results support the notion that free 
operant responding under VI schedules is 
organized in bouts separated by pauses. We 
effectively extended this notion to characterize 
extinction as an exponential lengthening of 
the pauses that separate bouts, which is 
consistent with prior findings (Podlesnik et 
al., 2006; Shull et al., 2002). Although we 
considered two models to account for mainte¬ 
nance and extinction performance (the static 
and dynamic models, respectively), both mod¬ 
els may be special cases of a more compre¬ 
hensive model. In this more general model, 
operant performance may be characterized as 
a propensity to initiate response bouts, which 
increases with reinforcement and declines with 
time. Alternatively, the decline in bout-initia¬ 
tion may be driven by unreinforced responses, 
in line with Skinner’s (1938/1991) notion of 
reflex reserve (Catania, 2005). More precisely, 
the model of bout-initiation decline consid¬ 
ered in this study was a hybrid of the time- 
dependent and response-dependent decline 
models: We assumed that bout initiations 
decline as time without reinforcement prog¬ 
ress, but the hypothesized clock was only 
updated with each response. Further research 
may clarify how the initiation of bouts declines 
in the absence of reinforcement. We also 
extended the bi-exponential model to include 
a refractory period following each response, 
and demonstrated that it is an informative 
component of operant performance. 

Although the refractory bi-exponential mod¬ 
el was capable of describing food-reinforced 
behavior in rats, it may be somewhat limited in 
describing the behavior of other species. 
Pigeon key-pecking has been notoriously 
resistant to bout-and-pause analyses. Visual 


inspection of log survival plots of key-pecking 
IRTs often fail to reveal a distinct inflection 
point (Bennett, Hughes & Pitts, 2007; Bowers, 
Hill & Palya, 2008; Podlesnik et al., 2006). This 
divergence in the performance of rats and 
pigeons suggests that a more flexible model, of 
which the refractory bi-exponential is a special 
case, may be necessary to characterize operant 
behavior across species. 

Motoric Effects 

When rats were required to press a higher, 
heavier lever, bout-initiation rate declined 
and the minimum time between consecutive 
responses (the refractory period § in Equa¬ 
tion 2) increased. Bout-initiation rate is also 
sensitive to deprivation level and rate of 
reinforcement (Shull, 2004; Shull et al., 
2004), but both were kept constant across 
workload manipulations. Thus, it appears 
that workload manipulations had an effect 
similar to that of altering deprivation and rate 
of reinforcement, aside from the more purely 
motoric effect of lengthening of the refrac¬ 
tory period. This is consistent with Posadas- 
Sanchez’s (2005) finding that a higher lever 
force requirement decreases indices of moti¬ 
vation while increasing indices of response 
duration. Skjoldager et al. (1993) reported 
that an increase in lever height and force 
requirement prolonged pre-ratio pauses and 
reduced run rates in a progressive ratio 
schedule. Ailing and Poling (1995) replicated 
these effects in fixed ratio schedules. Pre¬ 
ratio pauses are akin to between-bout IRTs— 
they indicate periods of disengagement from 
the operandum, and are particularly attuned 
to motivational manipulations such as those 
of deprivation level (Malott, 1966). All this 
evidence converges on the notion that chang¬ 
es in response cost necessarily influence 
motivation. Therefore motoric effects, which 
are embedded within changes in run rates, 
cannot be empirically isolated; they can only 
be analytically isolated. That is, motoric 
effects may not be observed without motiva¬ 
tional changes, but they may be estimated on 
the basis of changes in the shortest IRT. The 
refractory bi-exponential model provides a 
means for such estimation. 

The hypothesis that motoric manipulations 
imply motivational effects may explain some 
weaknesses of extant methods of motor- 
motivational dissociation. One popular meth- 
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od, for instance, consists of estimating the 
parameters of Herrnstein’s (1970) hyperbola 
from performance across various VI schedules 
(Bradshaw, Ruddle & Szabadi, 1981; Glautier, 
Rigney & Wilner, 2001; Heyman, Kinzie & 
Seiden, 1986). A common assumption, derived 
from early interpretations of the hyperbolic 
parameters (de Villiers & Herrnstein, 1976), is 
that the asymptotic response rate k attained 
with very high reinforcement rates should be 
sensitive only to motoric manipulations, 
whereas the rate of reinforcement r e that yields 
a response rate of k / 2 should only be sensitive 
to motivational manipulations. A recent review 
(Dallery & Soto, 2004) suggests that k is 
sensitive to motoric manipulations, but not 
selectively: It also appears to be sensitive to 
changes in deprivation level and reinforcer 
magnitude. In turn, r e seems to be sensitive to 
motoric manipulations under certain circum¬ 
stances. Motivational effects inherent to mo¬ 
toric manipulations may explain why r e is 
sensitive to motoric manipulations, but not 
why k is sensitive to motivational manipula¬ 
tions. A generalization of Herrnstein’s hyper¬ 
bola (McDowell, 2005), similar to that provid¬ 
ed by Baum (1974) for the matching law, 
appears to account for motivational effects 
otherwise absorbed by k. This may be a 
productive development toward a global ac¬ 
count of performance in VI schedules. Local 
mechanisms similar to those suggested here, 
however, are not specified by the generalized 
hyperbola. 

Schedule Effects 

Rats emitted fewer response bouts when a 
FR requirement was appended at the end of an 
interval schedule. These bouts, however, were 
longer and denser in responses (smaller q, 
higher w ), yielding higher overall response 
rates. If overall response rate was taken as an 
index of motivation, we would be deceived 
into believing that the imposition of a tandem 
FR schedule increased the motivation to 
engage the operandum. The reduction in 
bout-initiation rate suggests the contrary: The 
tandem FR schedule reduced, not increased, 
motivation. Why would a tandem FR schedule 
reduce motivation? Longer bouts increase the 
number of intervening responses between 
bout-initiating lever presses and reinforce¬ 
ment, and may widen the interval between 
the two. It is widely acknowledged that delayed 


reinforcement is less effective in maintaining 
behavior (e.g., Dickinson, Watt & Griffiths, 
1992). More directly relevant to our results, 
pre-ratio pauses in FR schedules increase with 
reinforcement delay (Meunier, Starratt & 
Sergio, 1979; Morgan, 1972). Although the 
ineffectiveness of delayed reinforcement has 
a credit-assignment component (Lieberman, 
McIntosh & Thomas, 1979), it is likely to 
include also a motivational component: De¬ 
layed incentives are less attractive than imme¬ 
diate ones (e.g., Mazur, 2010). Bout initiation 
rate may thus be reduced by tandem FR 
schedules because these schedules yield longer 
bouts that separate their initiation from 
reinforcement. This explanation has a sig¬ 
nificant implication: It suggests that, even 
though the computer recording the rat’s 
activity may count a bout-initiation lever 
press just as any other lever press, these lever 
presses are functionally distinct from within- 
bout lever presses. Despite their topographical 
similarity, bout-initiating responses and within- 
bout responses may constitute separate re¬ 
sponse classes (Schick, 1971). If such were the 
case, reinforcement of the latter would not 
completely generalize to the former, and vice 
versa (see Shull et al., 2004, pp. 76-78, for a 
related discussion on the bout as a behavioral 
unit). 

Our explanation of reduced bout-initiation 
rates in tandem FR schedules is based on 
longer bouts. Why, then, do tandem FR 
schedules yield longer bouts? Probably be¬ 
cause fast response bursts are more likely to be 
reinforced under tandem VT FR than under 
VI schedules (Killeen, 1969); schedules with a 
terminal tandem FR requirement selectively 
reinforce longer bursts. Interestingly, such 
reinforcement did not appear to generalize 
to bout-initiation lever presses in the present 
study, supporting the notion that bout-initia¬ 
tion and within-bout lever presses are func¬ 
tionally distinct. 

Like workload manipulations, schedule ma¬ 
nipulations appear to have a motivational and 
a nonmotivational component. The motiva¬ 
tional component is related to the resulting 
delay between bout initiation and reinforce¬ 
ment. The nonmotivational component is 
related to the proximity of within-bout re¬ 
sponses to reinforcement. The refractory bi¬ 
exponential model identifies changes in the 
former component with changes in parameter 
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b in Equation 2, and changes in the latter 
component with changes in q and w. 

Motivational Effects 

Food deprivation selectively increased bout- 
initiation rate (Phase 1), whereas the elimina¬ 
tion of reinforcement selectively reduced bout- 
initiation rate (Phase 2). Similarly, Podlesnik et 
al. (2006) found that prefeeding and extinction 
reduced bout-initiation rate. The results report¬ 
ed here complement the evidence that bout- 
initiation rate selectively covaries with rate of 
reinforcement (Shull et al., 2001). They suggest 
that the rat’s willingness to engage the oper- 
andum, which is driven by hunger, reinforcer 
availability, and low “price”, is expressed in the 
rate of bout initiation (parameter b in Equation 
2). Such general willingness may be described 
as operant motivation, to distinguish it from the 
more specific notion of incentive motivation 
(Bindra, 1978). 

Incentive motivation refers to the approach 
or seeking behavior elicited by appetitive or 
conditional stimuli. Incentive motivation is 
mostly dependent on the properties of the 
stimulus and on the state of the animal with 
respect to the stimulus, and is independent of 
response cost and the interval between re¬ 
sponse and reinforcer. Operant motivation, 
instead, is a function of both incentive and 
response. Response cost may influence oper¬ 
ant motivation, as shown by the effects of 
motoric manipulations, and longer response- 
reinforcer intervals are likely to reduce oper¬ 
ant motivation, as inferred from the effects of 
schedule manipulation and rate of reinforce¬ 
ment. Incentive motivation for food, for 
instance, may be raised by depriving an animal 
of food or by presenting food (or associated 
stimuli) to the animal; operant motivation for 
engaging in food-producing activities may be 
raised by increasing incentive motivation for 
food, by reducing the energetic cost of the 
activities that yield food, or by reducing the 
time between activities and food. Because 
incentive motivation is subsumed within oper¬ 
ant motivation, changes in incentive motiva¬ 
tion may be inferred from concomitant chang¬ 
es in bout-initiation rate, but only when 
response-reinforcement contingencies are 
kept constant. 

Researchers are often interested in drawing 
inferences about changes in incentive motiva¬ 
tion from operant performance. We may want 


to demonstrate, for instance, that a particular 
treatment reduces the incentive motivation for 
cocaine. One way to perform this demonstra¬ 
tion is to compare operant performance for 
cocaine (under maintenance or extinction 
schedules) with and without the treatment. 
Based on the results reported here, one would 
be advised not to compare overall response 
rates, but to compare estimates of the rate at 
which response bouts are initiated. Even then, 
because bout initiation is an indication of 
operant, not incentive motivation, one would 
have to rule out motoric and schedule effects 
that would indirectly affect operant motiva¬ 
tion. That is, if the test indicates that only 
bout-initiation rates were affected by treat¬ 
ment, it would constitute positive evidence 
that the treatment reduced the incentive 
motivation for cocaine, but if it also affected 
the within-bout response rate or the minimum 
IRT, reliable inferences on incentive motiva¬ 
tion may not be drawn. 

The previous example assumes that the 
inferences we have drawn from food-seeking 
behavior may be generalized to cocaine self¬ 
administration. Whether such generalization is 
justified or not will require further research 
with a wider range of reinforcers (e.g., drugs, 
access to mates, defense against aversive 
stimuli) and a variety of operants. The present 
study has laid out the empirical and analytical 
methods to assess changes in the components 
of operant performance, even when the 
reinforcer is absent. We believe these methods 
are critical to advance our understanding of 
motivated behavior. 
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2009; Sanabria, Acosta, Killeen, Neisewander 
& Bizo, 2008; Sanabria & Killeen, 2008). The 
AIC for a model is computed as AIC = 2k — 
21n ( MLE) , where k is the total number of free 
parameters, i.e., the number of parameters 
allowed to vary in the model, multiplied by the 
number of subjects. For example, if q, w, b, and 
§ are allowed to vary freely for 10 subjects, It = 
4X10 = 40. The model with the lowest AIC 
(AICmin) represents the best balance between 
likelihood (high MLE) and parsimony (low k). 
AAIC was computed for model i as AAICj = 
AIC; — AICmin- As a rule of thumb, if AAIC, > 
4 the evidence for model i is considered weak 
relative to the model with the lowest AIC (the 
best fitting model). This is because the 
likelihood of model i relative to the best fitting 
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model is exp(AAIC,/2) (Anderson & Burn¬ 
ham, 2002), and with AAIQ > 4, it at least g 2 ~ 
7 times more likely to observe the data using 
the best fitting model than using model i. 
Following this rule, the simplest model (lowest 
k) with AA1C, < 4 was favored. 

APPENDIX B 

Probability of the interval between the last 

RESPONSE IN A SESSION AND THE END OF THE SESSION 

Let us assume that the animal emitted its last 
response of the session at time L, and that the 
session ended at time S. Then, under the 
assumption of the dynamic refractory bi¬ 
exponential model, the probability that an 
animal emits no responses between L and S is: 

p (no response between L and S\S— L<8) = 1 

p (no response between L and S\S — L>5) = 

(1 - q)e-^ s ~ L -^ + qe- b ^ s - L - & \ (Al) 


For nonrefractory models, fix § at zero. For 
single exponential models, fix q at zero. 

APPENDIX C 

Refractory bi-exponential model parameters 

The following tables show estimates of q, 5, 
w, and b in separate tables. Estimates are 
shown separately for each individual rat in 
each experimental condition. Estimates were 
obtained for each daily session. For the VI and 
Tandem conditions, the mean estimate of the 
last four sessions is shown. 


Table Cl 

Estimates of parameter q (probability of quitting a response bout). 


Rat 

Low workload lever 


High workload lever 

VI 

Tandem 

Food Dep 

VI 

Tandem 

Food Dep 

505 

0.52 

0.16 

0.12 

0.11 

0.09 

0.08 

507 

0.26 

0.14 

0.14 

0.25 

0.13 

0.15 

517 

0.48 

0.15 

0.17 

0.32 

0.13 

0.13 

519 

0.43 

0.24 

0.20 

0.73 

0.14 

0.14 

520 

0.67 

0.12 

0.07 

0.75 

0.13 

0.10 

521 

0.47 

0.16 

0.15 

0.38 

0.13 

0.13 



Table C2 

Estimates of parameter 5 (minimum IRT duration) in 

seconds. 




Low workload lever 



High workload lever 


Rat 

VI 

Tandem 

Food Dep 

VI 

Tandem 

Food Dep 

505 

0.08 

0.08 

0.06 

0.10 

0.09 

0.07 

507 

0.09 

0.07 

0.09 

0.11 

0.07 

0.08 

517 

0.08 

0.08 

0.08 

0.06 

0.13 

0.16 

519 

0.18 

0.10 

0.06 

0.27 

0.08 

0.06 

520 

0.18 

0.10 

0.09 

0.28 

0.09 

0.08 

521 

0.06 

0.06 

0.06 

0.17 

0.09 

0.09 
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Table C3 

Estimates of parameter w (within-bout response rate) in responses per second. 


Rat 

Low workload lever 


High workload lever 

VI 

Tandem 

Food Dep 

VI 

Tandem 

Food Dep 

505 

2.07 

3.31 

3.48 

2.34 

4.69 

3.17 

507 

1.91 

2.91 

3.16 

1.71 

3.79 

3.54 

517 

5.31 

3.11 

2.59 

3.14 

8.60 

17.13 

519 

1.06 

4.03 

4.48 

4.17 

5.69 

5.43 

520 

1.15 

3.90 

3.14 

2.52 

3.09 

3.17 

521 

2.27 

6.09 

7.30 

3.90 

9.56 

9.98 



Table C4 

Estimates of parameter b (rate of bout initiation) 

in responses per second. 




Low workload lever 



High workload lever 


Rat 

VI 

Tandem 

Food Dep 

VI 

Tandem 

Food Dep 

505 

0.20 

0.13 

0.19 

0.11 

0.06 

0.15 

507 

0.34 

0.19 

0.23 

0.11 

0.18 

0.21 

517 

0.17 

0.08 

0.24 

0.12 

0.06 

0.09 

519 

0.14 

0.11 

0.28 

0.05 

0.10 

0.33 

520 

0.08 

0.03 

0.05 

0.04 

0.02 

0.03 

521 

0.25 

0.23 

0.33 

0.18 

0.11 

0.23 


APPENDIX D 

Probability of the interval between the last 

RESPONSE IN A SESSION AND THE END OF THE SESSION, 
ACCORDING TO THE DYNAMIC MODEL 

Let us assume that the animal emitted its last 
response of the session at time L, and that the 
session ended at time S. Then, under the 
assumption of the dynamic refractory bi¬ 
exponential model, the probability that an 


animal emits no responses between L and S is: 
p (no response between L and S\S—L<8) = 1 
p (no response between L and S\S—L>8) = 

(1 - q L )e-^ s ~ L -^ + q L e- bL{S - L - S) , (A2) 

where q L , w L and b L are calculated by substi¬ 
tuting L into t in Equation 5. For nonrefrac¬ 
tory models, fix § at zero. For single exponen¬ 
tial models, fix q at zero. 














